A common architecture for enterprise computing systems includes a client computer and a storage system, where the client system performed most of the computational intensive tasks using applications programs and the information needed for the computation was retrieved from a storage system. Often the storage system was not directly attached to the computer. The connection between the two system components depended on the data storage concept and was often described as a SAN (storage attached network) where data was stored as fixed sized blocks and as a NAS (network attached storage) where data was stored as files.
The storage component has been typically comprised of a server computer and a plurality of hard disk drives for the actual data storage. Files are often used for unstructured data such as images, text and the like whereas block storage has been associated with, for example, database processing. In all of these applications, the response time of the storage component of a computing system has been limited by the mechanical performance of the associated disk drives. Mixed workloads, a combination of block and file applications, were avoided due to unfavorable interactions between the access requirements.
Flash memory arrays are replacing disk storage devices in many applications due to the more rapid response time to client requests for reading and writing data as well as a capability to perform a much higher number of input/output (I/O) operations per second. However, at present, the hardware cost of the flash memory is greater than that of disk and the flash memory is perceived to have a wear out problem, at least if not properly managed. The amount of data to be stored on a global basis appears to grow substantially, despite all efforts to restrict this growth, and more efficient data storage techniques have been developed using techniques such as data de-duplication and compression. With disk storage systems, post-storage deduplication may be performed; the data to be written to the storage device is usually first written to a disk and subsequently de-duplicated so as to optimize disk performance. Often the deduplication is not performed until the data is backed up; the process is costly both computationally and in terms of volatile metadata storage. The wear-out characteristics of flash memory have led to the performance of these data reduction techniques in-line in order to reduce the amount of data that is actually stored.
Data de-duplication is effective for certain application types such as email attachments, operating system or virtual desktop images and the like, and is usually less effective for data base information. Similarly, data compression is effective for images, text files and the like, but relatively ineffective for encrypted or already compressed data.
System architecture is also influenced by the virtualization of computing systems, where server computers are being configured to support the effectively simultaneous operation of a plurality of application programs, each with its own allocation of computing and memory resources. This trend results in a fragmentation of the temporal pattern of storage operations as the independent storage system requests from the application programs are contemporaneously presented to the storage devices from a plurality of using applications.
FIG. 1 (prior art) shows a top level block diagram of a virtualized computing system 1 configured to service guests (users) or applications representing mixed data storage requirements. That is, some of the guests may store data as files using block storage and communicating with a memory system 2 through a SAN network that supports an appropriate communications protocol. Guests that store data using a file-oriented approach such as NFS may communicate with memory system 3 through a NAS network that supports an appropriate communications protocol. Other protocols communications and file access may be used and are often proprietary to a vendor and have differing levels of functionality.
Converged storage architectures are being developed that support virtualized users having a mixture of block and file-based storage requirements either individually or collectively. The server and flash memory array may therefore need to provide the storage required for both types of data, depending on the application programs, while maintaining the consistently low latency and high-I/O-rate capability that are the salient requirements of the evolving data storage systems.